2). ExVivo_InVivo_Registration Pipeline

1. About the example data

The example data in ExVivo_InVivo_Reg\ZStack_XDays subfolder are the slow zstack images aquired in meta trials. These slow zstack images are used for ex vivo to in vivo registration and image registration across days. The organization of these images are shown below.
Except for d10-Perfusion, the subfolder names are in dn-xxxx format , where n inidcates the nth day of the whole imaging experiments and xxxx is the imaging experiment related behavior. ANM378231_d10-Perfusion.tif in d10-Perfusion folder is the image zstack acquired through GRIN lens after perfusion. Except for \d6-Fear\ANM378231_d6-Fear_Full.tif and \d10-Perfusion\ANM378231_d10-Perfusion.tif, other images are acquired in the same imaging depth ranges as those in fast-z images. \d6-Fear\ANM378231_d6-Fear_Full.tif and \d10-Perfusion\ANM378231_d10-Perfusion.tif are acquired in larger imaging depth range.
Note1: For each image file, there is a corresponding .mat file as shown in gray below. The .mat file provides the original Scanimage meta information of the image zstacks, which is useful for the reconstruction of 2p image 3D volume in next section.
Note2: Following MIP_Z_nn-mm images are the maximum intensity projection of images on day 1, 6 and 10, where nn and mm indicate the starting and end z depth of the projection volumn. MIP_Z_1-15, MIP_Z_33-47 and MIP_Z_60-74 compare the neuronal structure in superficial, middle and deep layer of the imaging volumn on different days respectively. Comparison of these slow zstack images showed that the FOV of imaging brain region was quite stable during the whole imaging experiments.
Note3: The neuronal image of the brain region under GRIN lens largely changed after perfusion. As shown below, after perfusion, although we can find many corresponding neurons, as marked with the same color arrows, the imging FOV shifted down and the fluorescence intensities of some neurons, such as those pointed by the red and green arrows, dramatically changed. The image zstack after perfusion provides a crucial intermediate guidance between the ex vivo images and in vivo 2p images acquired in behavioral experiments.

2. Reconstruct the 2p image 3D volume by correcting the FOV curvature, remapping the objective-sample and objective-magnification relationships

As shown below (from the figS.3 in our paper 1), the two-photon-GRIN-lens imaging system has nonlinear objective-sample relationship (H), objective-position-dependent magnification (I) and FOV curvature (J). To find the same neurons in in vivo imaging volume and ex vivo stack, we need to reconstruct the 2p image 3D volume by correcting the FOV curvature, remapping the objective-sample and objective-magnification relationships using the calibrated optical properties of the two-photon-GRIN-lens imaging system.
The optical properties measured with 2p excitable GRID target are stored in \ExVivo_InVivo_Reg\FOVs.mat. There are 5 variables in the file: GRID_SPACE, vtPosZ, matCt_Md, clFits and clFOVs. Details about these variables are in \ExVivo_InVivo_Reg\FOVs_readme.txt.
Batch_ReconstuctImage3D
Note1: I left these reconstructed images in the example data folder so that you can examine the correction changes.

3. Stitch image stacks of ex vivo brain slices using the anatomy landmarks and under the guidance of the reconstructed 2p image 3D volume

1. Coarsely stitch consective brain sections in TrakEM2 using the GRIN lens hole and anatomy landmarks, such as 3V.

a). Import MIP images of consective brain sections into TrakEM2 in FiJi as shown below

b). Manually transform (translate and rotate) the neigbor images so that they are well aligned with anatomy landmarks

Note: the live filter as shown above can be helpful in the coarse alignment processing.

2. Perform fine stitching under the guidance of the reconstructed 2p image 3D volume.

ANM378231_Confocal_MIPs.tif in \ExVivo_InVivo_Reg\Reg\Reg_Demo subfolder contains coarsely aligned MIPs of 3 consecutive sections.

a). Find neurons with distinct features in the reconstructed 2p image volume and ex vivo images of brain sections

As shown below, 11 neurons with distinct features could be easily identified in ANM378231_d10-Perfusion_Reconstructed.tif and ANM378231_Confocal_MIPs.tif. These neurons will be used as fiducial markers in next steps.
Note: the images from Confocal_MIPs.tif above were zoomed-in subregion of ANM378231_Confocal_MIPs.tif.

b). Manually adjust the neigbor images under the guidance of the reconstructed 2p image 3D volume so that the stitching precision < 10 um (about 2/3 neuron size)

Below shows the relative locations of those 11 fiducial neurons in corrected 2p image volume. Image at Z=73 is green and image at Z=89 is red.
As shown below, the coarsely aligned images ANM378231_Confocal_MIPs.tif provides largely right location of those neurons. However, the relative positions of those fiducial neurons were mildly messed up compared with those in reconstructed 2p image volume. For example, the relative locations among neuron 1, 8 and 9, relative locations among neuron 5, 6 and 7 are different from those in intact 2p volume.
With the guidance of the reconstructed image volume, we can manually adjust (translate and rotate) the neigbor images so that the relative positions of those fiducial neurons are almost the same as those in reconstructed 2p image volume. Below shows the well stitched images of ANM378231_Confocal_MIPs.tif.

4. Find the corresponding neurons in in vivo 2p image volume and ex vivo confocal image stack using TrakEM2

After stitching ex vivo images, we used these identified fiducial neurons as starting points to find their neighbor neurons matched between the corrected ZStack images in fear retrieval experiment (middle of the entire imaging experiments) and ex vivo images.
2P_Fear.xml and Confocal_MIPs.xml in \ExVivo_InVivo_Reg\Reg\Reg_Demo subfolder are TrakEM2 projects demonstrating the ex vivo to in vivo registration with 40 matched neurons in a subset of the 2p imaging volume.
2P_Neurons_8P.xml in \ExVivo_InVivo_Reg\Reg\Reg_All_Planes is a TrakEM2 project with 99 neurons identified in ex vivo images in all 8 imaging planes.
Note1: to facilitate observing the dark neurons, the gamma of the images imported into 2P_Neurons_8P.xml was adjusted to 0.45.
Note2: because of the relative elongated axial resolution (~10 µm) of the two-photon-GRIN-lens imaging system, the images of some neurons may be spatially overlapped as shown above.

5. Export ROIs of individual neurons from TrakEM2

Because the images of individual neurons can be spatially overlapped and across multiple imaging planes, each neuron was exported as a 3D ROI mask.
Save_AreaList_As_ROIs.py in \Analysis_Pipeline\Local\ExVivo_InVivo_Reg subfoler was used to export ROIs of individual neurons from TrakEM2. To run this Jython script in FiJi, you need to add FiJi-Legacy to your update sites and initiate Jython interpreter in FiJi plugin as shown below.
Save_AreaList_As_ROIs.py %in FiJi
Two variables savDir and Label_Max need to be set. Please read the details about them in the script. The script generates a Tiff stack for each ROI. The file name of the Tiff stack is in Niii format as shown below, where iii is the ROI ID in TrakEM2.

6. Register images from fastZ to slow zstack

During fastZ volumetric imaging, the image size was set to 256x256 to increase the temporal resolution. However, in slow zstack imaging, to increase the spatial resolution, the image size was set to 512x512. Beside the difference in image size, we found during fastZ imaging, there are slight and consistent mechanical vibrations of the objective lens during the fast movement.
In subfolder \ExVivo_InVivo_Reg\Reg\ZStack_FastZ_Reg, ANM378231_00004_Stack_C1_reg_Avg_2X.tif is the 2x template images in individual fastZ imaging planes and ANM378231_00002_reg_ZProj_ZStack_Correct_Avg8P is the average images of slow zstack in the same imaging ranges as those during fastZ imaging.
As shown below, by comparing these two image stacks, you can find the vibrations in fastZ image (ANM378231_00004_Stack_C1_reg_Avg_2X.tif) and the stability in slow zstack (ANM378231_00002_reg_ZProj_ZStack_Correct_Avg8P). Because of the stability in slow zstack, we use the slow zstack to do the ex vivo to in vivo registration. To generate the ROIs for fastZ imaging, we need to register the fastZ image stack to slow zstack.
The script below was run in Janelia Computer Cluster to split the image stacks into images of individual imaging planes and register the images in fastZ to their corresponding images in slow zstack.
MatlabCmd_Reg_StackZ_FastZ
For the example data, the script finally generates the registered fastZ image stack as highlighted in blue below and saves the transformation information in subfolder \ExVivo_InVivo_Reg\Reg\ZStack_FastZ_Reg\ANM378231_00004_Stack_C1_reg_Avg_2X\ImgSeq as shown below.
Below is the overlay of registered fastZ image stack and slow zstack. The registration aligns neuronal structure very well, although their intensities may be different.

7. Transform ROIs to fast-z images

Using the script below, the ROIs, which were identified during ex vivo <-> in vivo registration in zstack, were transformed to fastZ imaging planes with the transformations above. The script was also implemented to run in Janelia Computer Cluster.
Batch_MatlabCmd_ApplyTrans_RegROIs_Inverse_AllNeurons
The transformed ROIs are in \Fear_Imaging_Exp\2P_Imaging\Avg3\ROIs_Reg and can be used to extract calcium dynamic responses from images as shown in TwoPhoton_Imaging_Processing_Pipeline.mlx.

7. Transform ROIs from reference day (fear) to other days

To reduce the time of manually drawing ROIs, we transformed the ROIs manually drawn in Fear experiment to other experiments.
In subfolder \ExVivo_InVivo_Reg\Reg\ZStacks_Xdays_Reg, ANM378231_00002_reg_ZProj_ZStack_Correct_Avg8P_Fear.tif and ANM378231_00002_reg_ZProj_ZStack_Correct_Avg8P_Ghrelin.tif are the average images of slow zstack in the same imaging ranges of the 8 imaging planes in fastZ imaging from Fear and Ghrelin injection experiments, respectively.
The script below was run in Janelia Computer Cluster to split the image stacks into images of individual imaging planes and register the corresponding images across days.
MatlabCmd_Reg_StackZ8P_Xdays
For the example data, the script finally generates the registered image stack for Fear experiment as highlighted in blue below and saves the transformation information in the subfolder \ExVivo_InVivo_Reg\Reg\ZStacks_Xdays_Reg\ANM378231_00002_reg_ZProj_ZStack_Correct_Avg8P_Fear\ImgSeq as shown below.
Below shows the overlay of images from Fear and Ghrelin Experiments before and after registration.
The script below was run in Janelia Computer Cluster to transform ROIs from reference day (fear in the subfolder \ExVivo_InVivo_Reg\Reg\ZStacks_Xdays_Reg\2p_ROIs_Fear) to other days using the transformation information in the subfolder \ExVivo_InVivo_Reg\Reg\ZStacks_Xdays_Reg\ANM378231_00002_reg_ZProj_ZStack_Correct_Avg8P_Fear\ImgSeq
MatlabCmd_ApplyTrans_RegROIs_CrossDays
The transformed ROIs are in \ExVivo_InVivo_Reg\Reg\ZStacks_Xdays_Reg\2p_ROIs_Fear with _reg.tif suffix as highlighted in blue below

8. Import the transformed ROIs for other days overlay with corresponding images into TrakEM2

Create a new TrakEM2 project using \Analysis_Pipeline\Local\ExVivo_InVivo_Reg\neuron.dtd as template and import \ExVivo_InVivo_Reg\Reg\ZStacks_Xdays_Reg\ANM378231_00002_reg_ZProj_ZStack_Correct_Avg8P_Ghrelin.tif as shown below
Import_ROIs_AreaLists_v3.py in \Analysis_Pipeline\Local\ExVivo_InVivo_Reg subfoler was used to import ROIs of individual neurons to TrakEM2. To run this Jython script in FiJi, you need to add FiJi-Legacy to your update sites and initiate Jython interpreter in FiJi plugin as shown in 5. Export ROIs of individual neurons from TrakEM2.
Import_ROIs_AreaLists_v3.py %in FiJi
Two variables strDir and nROIsCount need to be set. Below shows the overlay of imported ROIs and images from Ghrelin experiment

9. Manually correct the imported ROIs in TrakEM2

The imported ROIs were visually inspected and manually corrected because of the somehow Z shift across days.
The manually corrected ROIs were exported and registered to fastZ as in 5. Export ROIs of individual neurons from TrakEM2 and 6. Register images from fastZ to slow zstack.